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Widespread vaccination remains the best option for controlling the spread of COVID-19 and ending the 
pandemic. Despite the considerable disruption the virus has caused to people’s lives, many people are 
still hesitant to receive a vaccine. Without high rates of uptake, however, the pandemic is likely to be pro- 
longed. Here we use two survey experiments to study how persuasive messaging affects COVID-19 vac- 
cine uptake intentions. In the first experiment, we test a large number of treatment messages. One 
subgroup of messages draws on the idea that mass vaccination is a collective action problem and high- 
lighting the prosocial benefit of vaccination or the reputational costs that one might incur if one chooses 
not to vaccinate. Another subgroup of messages built on contemporary concerns about the pandemic, like 
issues of restricting personal freedom or economic security. We find that persuasive messaging that 
invokes prosocial vaccination and social image concerns is effective at increasing intended uptake and 
also the willingness to persuade others and judgments of non-vaccinators. We replicate this result on 
a nationally representative sample of Americans and observe that prosocial messaging is robust across 
subgroups, including those who are most hesitant about vaccines generally. The experiments demon- 
strate how persuasive messaging can induce individuals to be more likely to vaccinate and also create 
spillover effects to persuade others to do so as well. 

The first experiment in this study was registered at clinicaltrials.gov and can be found under the ID 
number NCT04460703. This study was registered at Open Science Framework (OSF) at: https://osf.io/q 
u8nb/?view_only=82f06ecad77f4e54b02e8581a65047d7. 

© 2021 Elsevier Ltd. All rights reserved. 


1. Introduction of a COVID-19 vaccine, it has become increasingly important to 


understand how public health communication can play a role in 


The global spread of COVID-19 created an urgent need for safe 
and effective vaccines against the disease. However, even though 
several successful vaccines have become available, vaccine hesi- 
tancy in the general population has the potential to limit the effi- 
cacy of vaccines as a tool for ending the pandemic. For instance, 
in the United States, the public’s willingness to receive a vaccine 
has declined from 72 % saying they would be likely to get a 
COVID-19 vaccine in May 2020 to 60 % of people reporting that 
they would receive a vaccine as of November 2020 [1]. Given the 
considerable amount of skepticism about the safety and efficacy 
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increasing COVID-19 vaccine uptake. 

Vaccination is both a self-interested and a prosocial action [2- 
9]. By getting vaccinated, people protect themselves from a dis- 
ease, but they also reduce the chance that they become a vector 
through which the disease spreads to others. If enough people 
receive a vaccine, the population gains protection through herd 
immunity, but this also creates an incentive for an individual to 
not get vaccinated because they can forgo vaccination and receive 
protection from others who do vaccinate. Recent research on vac- 
cination in general has demonstrated that people view vaccination 
as a social contract and are less willing to cooperate with those 
who choose not to get inoculated [10]. This work also implies that 
highlighting the reputational costs of choosing not to vaccinate 
could be an effective strategy for increasing uptake. Further, 
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appeals to herd immunity and the prosocial aspect of vaccination 
have been shown to increase uptake intentions [11-13], but 
emphasizing the possibility of free riding on other’s immunity 
reduces the willingness to get vaccinated [14]. 

Focusing specifically on vaccination against COVID-19, recent 
studies have found that messages that explain herd immunity 
increase willingness to receive a vaccine [15] and reduces the time 
that people would wait to get vaccinated when a vaccine becomes 
available to them [16]. However, other work has found that proso- 
cial appeals did not increase average COVID-19 vaccination inten- 
tions [17] and the effect of prosocial concerns was present in 
sparsely populated places, but absent in more densely populated 
ones [18]. Given the current state of evidence, it is unclear whether 
appealing to getting a COVID-19 vaccine as a way to protect others 
will increase willingness to vaccinate. 

Viewing vaccination through the lens of a collective action 
problem suggests that in addition to increasing individuals’ inten- 
tions to receive a vaccine, effective public health messages would 
also increase people’s willingness to encourage those close to them 
to vaccinate and to hold negative judgments of those who do not 
vaccinate. By encouraging those close to them to vaccinate, people 
are both promoting compliance with social norms and increasing 
their own level of protection against the disease. Also, by judging 
those who do not vaccinate more negatively, they apply social 
pressure to others to promote cooperative behavior. This would 
be consistent with theories of cooperation, like indirect reciprocity 
or partner choice, that rely on free riders being punished or ostra- 
cized for their past actions to encourage prosocial outcomes [19- 
23]. Thus, effective messaging could have outsized effects on pro- 
moting vaccination if it both causes people to vaccinate themselves 
and to encourage those around them to do so. 

We conducted two pre-registered experiments to study how 
different persuasive messages affect intentions to receive a 
COVID-19 vaccine, willingness to persuade friends and relatives 
to receive one, and negative judgments of people who choose not 
to vaccinate. In the first experiment, we tested the efficacy of a 
large number of messages against an untreated control condition 
(see Table 1 for full text of messages). A subgroup of the messages 
in Experiment 1 drew on this collective action framework of vacci- 
nation and emphasized who benefits from vaccination or how 
choosing not to vaccinate hurts one’s social image. A second sub- 
group drew on contemporary arguments about restrictions on lib- 
erty and economic activity during the COVID-19 pandemic. In 
Experiment 2, we retested the most effective messages from 
Experiment 1 on a nationally representative sample of American 
adults. By utilizing this test and re-test design, we guard against 
false positive results that are observed by chance among the large 
number of messages tested in Experiment 1. In our analysis of both 
experiments, we examined whether specific messages were more 
effective among certain subgroups of the population. 

Experiment 1 was fielded in early July 2020. Participants were 
randomly assigned to either a placebo control condition in which 
they read a story about the effectiveness of bird feeders or one of 
eleven treatment messages. The first message is a Baseline infor- 
mational control condition that describes how it is important to 
receive a vaccine to reduce your risk of contracting COVID-19 or 
spreading it to others. Informational messages have been shown 
to be effective at increasing COVID-19 vaccine uptake intentions 
[24]. This message also emphasized that vaccines are safe and esti- 
mated to save millions of lives per year. The other messages add 
additional content to this baseline message. 

The subgroup of messages that emphasized collective action 
varied who would benefit from vaccination or what other people 
might think of someone who chooses to be a free rider by not vac- 
cinating. Focusing on who benefits from vaccination, the second 
message invoked Self Interest and reinforced the idea that vaccina- 
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tion is a self-protecting action (“Remember, getting vaccinated 
against COVID-19 is the single best way to protect yourself from 
getting sick.”). The third message, Community Interest, instead 
argued that vaccination is a cooperative action to protect other 
people (“Stopping COVID-19 is important because it reduces the 
risk that members of your family and community could get sick 
and die.”). This message also invoked reciprocity by emphasizing 
the importance of every-one working together to protect others. 

The fourth, fifth, and sixth messages added an invocation of an 
emotion, Guilt, Embarrassment, or Anger, to the Community Inter- 
est message. These messages prompted people to think about how 
they would feel if they chose not to get vaccinated and spread 
COVID-19 to someone else in the future. Emotions are thought to 
play a role in cooperation, either by motivating an individual to 
take an action because of a feeling that they experience or restrain- 
ing them from taking an action because of the emotional response 
it would provoke in others [25-27]. Further, anticipated emotional 
states have been shown to promote various health behaviors, like 
vaccination [28,29]. 

The seventh and eighth messages evoked concerns about one’s 
reputation and social image, which influences their attractiveness 
as a cooperative partner to others. The seventh, a Not Bravery mes- 
sage, reframed the idea that being unafraid of the virus is not a 
brave action, but instead selfish, and that the way to demonstrate 
bravery is by getting vaccinated because it shows strength and 
concern for others (“To show strength get the vaccine so you don’t 
get sick and take resources from other people who need them 
more”). The eighth message was a Trust in Science message that 
highlights that scientists believe a vaccine will be an effective 
way of limiting the spread of COVID-19. This message suggests that 
those who do not get vaccinated do not understand science and 
signal this ignorance to others (“Not getting vaccinated will show 
people that you are probably the sort of person who doesn’t under- 
stand how infection spreads and who ignores or are confused 
about science.”). 

The final three messages drew on concerns about restrictions on 
freedom and economic activity that were widespread during the 
COVID-19 pandemic. A pair of messages focused on how vaccina- 
tion would allow for a restoration of Personal Freedom (“Govern- 
ment policies to prevent the spread of COVID-19 limit our 
freedom of association and movement”) or Economic Freedom 
(“Government policies to prevent the spread of COVID-19 have 
stopped businesses from opening up”). These messages take a 
value that is commonly invoked in individuals’ decision to not vac- 
cinate [30,31] and reframed vaccination as something that would 
actually restore freedoms that had been taken away. The final mes- 
sage, Community Economic Benefit, argues that a vaccine will help 
return people’s financial security and strengthen the economy This 
message is similar to the Community Interest messages that are 
described above, but instead focuses on cooperating to restore 
the economy (“We can all end this outbreak and strengthen the 
national economy by working together and getting vaccinated”). 


2. Results 
2.1, Experiment 1 results 


Panel A of Fig. 1 plots the effect of each vaccine message relative 
to the untreated control group on intention to vaccinate. The inten- 
tion to vaccinate measure was formed by combining responses to a 
question about the likelihood of getting a COVID-19 vaccine within 
the first 3 months that one is available with a question about get- 
ting a vaccine within the first year that one is available. Specifi- 
cally, for respondents who did not answer that they were very 
likely to vaccinate within the first three months that a vaccine is 
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Table 1 
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Experimental treatment messages for Experiment 1 and Experiment 2. All messages add the prose in the table to the content of the Baseline informational control. All of the 
messages in the table were tested in Experiment 1. The messages that are bolded were retested in Experiment 2. 


Treatment Name 


Full Text 


= 


Baseline Informational 
Control 


N 


Self-Interest 


w 


Community Interest 


A 


Community Interest + Guilt 
Community 

Interest + Embarrassment 
Community Interest + Anger 
Not Bravery 


u 


n 


“I 


(8) Trust in Science 


(9) Personal Freedom 


(10) Economic Freedom 


(11) Community Economic 
Benefit 


To end the COVID-19 outbreak, it is important for people to get vaccinated against COVID-19 whenever a vaccine becomes 
available. Getting the COVID-19 vaccine means you are much less likely to get COVID-19 or spread it to others. Vaccines are safe 
and widely used to prevent diseases and vaccines are estimated to save millions of lives every year. 

Stopping COVID-19 is important because it reduces the risk that you could get sick and die. COVID-19 kills people of all ages, and 
even for those who are young and healthy, there is a risk of death or long-term disability. Remember, getting vaccinated against 
COVID-19 is the single best way to protect yourself from getting sick. 

Stopping COVID-19 is important because it reduces the risk that members of your family and community could get sick and die. 
COVID-19 kills people of all ages, and even for those who are young and healthy, there is a risk of death or long-term disability. 
Remember, every person who gets vaccinated reduces the risk that people you care about get sick. While you can’t do it alone, we 
can all protect every-one by working together and getting vaccinated. 

(3) + Imagine how guilty you will feel if you choose not to get vaccinated and spread COVID-19 to someone you care about. 

(3) + Imagine how embarrassed and ashamed you will be if you choose not to get vaccinated and spread COVID-19 to someone you 
care about. 

(3) + Imagine how angry you will be if you choose not to get vaccinated and spread COVID-19 to someone you care about. 
Soldiers, fire-fighters, EMTs, and doctors are putting their lives on the line to serve others during the COVID-19 outbreak. That’s 
bravery. But people who refuse to get vaccinated against COVID-19 when there is a vaccine available because they don’t think they 
will get sick or aren’t worried about it aren’t brave, they are reckless. By not getting vaccinated, you risk the health of your family, 
friends, and community. There is nothing attractive and independent-minded about ignoring public health guidance to get the 
COVID-19 vaccine. Not getting the vaccine when it becomes available means you risk the health of others. To show strength get the 
vaccine so you don’t get sick and take resources from other people who need them more, or risk spreading the disease to those who 
are at risk, some of whom can’t get a vaccine. Getting a vaccine may be inconvenient, but it works. 

Getting vaccinated against COVID-19 is the most effective means of protecting your community. The only way we can beat COVID- 
19 is by following scientific approaches, such as vaccination. Prominent scientists believe that once available, vaccines will be the 
most effective tool to stop the spread of COVID-19. The people who reject getting vaccinated are typically ignorant or confused 
about the science. Not getting vaccinated will show people that you are probably the sort of person who doesn’t understand how 
infection spreads and who ignores or are confused about science. 

COVID-19 is limiting many people’s ability to live their lives as they see fit. People have had to cancel weddings, not attend funerals, 
and halt other activities that are important in their daily lives. On top of this, government policies to prevent the spread of COVID- 
19 limit our freedom of association and movement. Remember, each person who gets vaccinated reduces the chance that we lose 
our freedoms or government lockdowns return. While you can’t do it alone, we can all keep our freedom by getting vaccinated. 
COVID-19 is limiting many people’s ability to continue to work and provide for their families. People have lost their jobs, had their 
hours cut, and lost out on job opportunities because companies aren’t hiring. On top of this, government policies to prevent the 
spread of COVID-19 have stopped businesses from opening up. Remember, each person who gets vaccinated reduces the chance 
that we lose our freedoms or government lockdowns return. While you can’t do it alone, we can all keep our ability to work and 
earn a living by getting vaccinated. 

Stopping COVID-19 is important because it is wreaking havoc on our economy. Thousands of people have lost their jobs and are 
unable to pay their bills. Many others have been laid off by their employers and do not know when they will be called to return to 
work. Remember, every person who gets vaccinated reduces the risk that someone else gets sick. While you can’t do it alone, we can 
all end this outbreak and strengthen the national economy by working together and getting vaccinated. 


available to them, we asked how likely they would be to vaccinate 
within a year. This measure coded those who are very likely in the 
first three months at the highest value on the scale followed by 
very likely within a year descending down to very unlikely within 
the first year. Analyzing the vaccination item separately does not 
substantively change the results. All outcome variables were 
scored 0 to 1, with higher values indicating greater willingness to 
endorse the pro-vaccine action or belief (Underlying regressions 
appear in Table S1 and unless otherwise noted, all analyses were 
pre-registered). 

Compared to the untreated control, the Baseline informational 
message was associated with modest increases in intention to vac- 
cinate by 0.034 units (95 % CI:0.002, 0.065; p < .05). This effect rep- 
resents an increase of approximately 6 % in the scale score 
compared to the outcome in the control condition. 

By comparison, the Community Interest, Community Inter- 
est + Guilt, Embarrassment, or Anger, Not Bravery, Trust in Science 
and Personal Freedom messages all produce larger effects, at least 
qualitatively, than the Baseline informational message on the 
intention to vaccinate outcome. Effects for the Self-Interest, Eco- 
nomic Freedom, and Community Economic benefit messages were 
not consistently distinguishable from the untreated control group 
outcomes, and their effects were indistinguishable from the effects 
of the Baseline informational message. 

The most promising messages were the Not Bravery, Commu- 
nity Interest, and Community Interest + Embarrassment messages. 
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These messages were associated with effects that were statistically 
distinguishable from the untreated control group (Not Bravery: 
0.077 units, 95 % CI: 0.035, 0.119; p < .01, Community Interest: 
0.090 units, 95 % CI: 0.050, 0.129; p < .01, Community Interest + 
Embarrassment: 0.094 units, 95 % CI: 0.054, 0.134; p < .01) at 
p < .01. Moreover, their effects were always more than twice as 
large as the Baseline informational treatment and these differences 
were significant at p < .05 (two-tailed tests). The effects of the Trust 
in Science message and the Personal Freedom message were not 
statistically significant when compared to the Baseline informa- 
tional message. 

To put the magnitudes of the effects into context, we re- 
estimated our analysis after dichotomizing the intended vaccine 
uptake measure such that those who report they were “somewhat” 
or “very” likely to get the vaccine, either with three months or a 
year, are coded as 1 and those who do not are coded 0 (this analysis 
was not pre-registered). This produced a predicted rate of intended 
vaccination in the control group of 58.2 %. Respondents who read 
the Baseline informational message were 7.4 percentage points 
(95 % CI: 2.9 pp, 12.0 pp; p < .01) more likely to receive a vaccine. 
Among those assigned to the Not Bravery or Community Interest 
messages it was predicted to increase by 10.4 percentage points 
and 12.7 percentage points (Not Bravery: 95 % CI: 4.3 pp, 
16.4 pp; p < .01, Community Interest: 95 % CI: 6.7 pp, 18.7 pp; 
p < .01) respectively, while among those assigned to the Commu- 
nity Interest + Embarrassment message it was predicted to 
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Fig. 1. Experiment 1. Messages that frame vaccination as a cooperative action to protect others or emphasize how non-vaccination might negatively affect one’s social image 
increase reported willingness to advise a friend, and judgment of non-vaccinators. Panel A displays treatment effects for the combined measure of intention to vaccinate, 
Panel B displays the advise a friend outcome, and Panel C displays the judging a non-vaccinator outcome. Treatment effects for both panels were estimated using OLS 
regression that included covariates. The effects displayed are a comparison against the placebo control baseline and are presented with 95% confidence intervals. The dashed 


vertical line is the effect of the Baseline informational control for each outcome. 


increases by 15.9 percentage points (95 % CI: 10.2 pp, 21.6 pp; 
p < .01). This last difference was substantively large, representing 
a proportional increase of 27 % (0.159/0.582) compared to the con- 
trol condition and a 13 % increase compared to the Baseline infor- 
mational condition (0.159-0.074)/(0.582 + 0.074). 

Turning to the other regarding outcomes that focused on spur- 
ring action by others, Panel B plots the effects of each vaccine mes- 
sage relative to the untreated control for advising a friend to 
receive a vaccine and Panel C plots the effects for negatively judg- 
ing someone who refuses to receive one. Here, the effect of the 
Baseline informational intervention was modest and statistically 
insignificant. However, the Not Bravery, Trust in Science, Personal 
Freedom, Community Interest, Community Interest + Guilt, and 
Community Interest + Embarrassment messages had larger effects 
on both outcomes that were statistically distinguishable from the 
control outcome. 

The most promising message was the Community Interest + E 
mbarrassment message for the advise a friend outcome, which 
was associated with a 0.09 unit increase in the scale outcome 
(95 % CI: 0.049, 0.132; p < .01 two-tailed test), an effect that repre- 
sents an increase of 27 % compared to the mean scale score in the 
control group. The effect was 0.067 units compared to the Baseline 
informational message (95 % CI: 0.027, 0.105; p = .001, two-tailed 
test). We conducted a similar exercise to the one describe above to 
gauge the relative magnitude of these treatment effects. For the 
Community Interest + Embarrassment message we estimated a 
15 percentage point increase (95 % CI: 0.088, 0.209; p < .01, two 
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tailed test,) in a binary intention to advise others to vaccinate out- 
come, a proportional increase of 27 % compared to the control 
group baseline of 53 % (0.15/0.53). This effect was also 6 percent- 
age points larger than the effect of the baseline message (95 % 
CI: 0.008, 0.121; p = .03, two-tailed test). 

The most promising outcome for the negative judgment of non- 
vaccinators was the Not Bravery message, which had an effect of 
0.09 scale points (95 % CI: 0.052, 0.126; p < .01, two-tailed test) 
compared to the untreated control and 0.072 scale points versus 
the Baseline information (95 % CI: 0.037, 0.106; p < .01 Baseline 
message, two-tailed tests). This corresponded to a 21 % increase 
compared to the scale outcome in the control group (0.09/0.43). 
These are both substantively and statistically meaningful effects. 
The Community Interest, Community Interest + Guilt, Community 
Interest + Embarrassment, Trust in Science, and Personal Freedom 
messages all produced effects that were statistically distinguish- 
able from the control condition. 

We also investigated the robustness of these findings to sample 
restrictions and whether certain subgroups were more responsive 
to specific treatment messages (reported in Figures S2-S12). 
Results were generally robust to restricting the sample to those 
who were over the 10th percentile and under the 90th percentile 
for completion time. For subgroup analyses, those scoring low in 
liberty endorsement appeared more responsive to the Baseline 
treatment and to the Not Bravery message than are those who 
scored high in liberty endorsement. Those who report being less 
likely to take risks appeared robustly more responsive to the Not 
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Bravery message than those who were high in risk taking. Those 
who were high in risk taking appear more responsive to the Per- 
sonal Freedom message with regard to their own behavioral inten- 
tions. Certain groups appeared generically easier to persuade 
(Democrats rather than Republicans, an important divide that 
has emerged during the pandemic [32], and Women rather than 
Men), but there were no clear differences in which treatments 
appeared most effective across these groups. We explored the 
robustness of these subgroup differences in Experiment 2. 

Taken together, the most successful messages in Experiment 1 
were those that were theoretically motivated by viewing vaccina- 
tion as a collective action problem. Consistent with previous work 
that demonstrates that prosocial appeals are effective in promoting 
vaccination, the Community Interest message and Community 
Interest + Guilt, Embarrassment, or Anger messages increased 
COVID-19 vaccine uptake intentions. Moving beyond who benefits 
from vaccination, the Not Bravery and Trust in Science messages 
that invoked concerns about one’s social image if they choose 
not to vaccinate also increased uptake intentions. All of the collec- 
tive action oriented messages increased intentions to advise a 
friend to vaccinate and negative judgments of those who do not, 
potentially creating spillover effects that induce others to vacci- 
nate. In addition to this subgroup of messages, we found that 
reframing vaccination as a way to restore freedom was also effec- 
tive, though the other messages motivated by contemporary 
debates about the pandemic were generally no more effective than 
the Baseline condition. 


2.2. Experiment 2 results 


Experiment 2 tested the subset of the best performing messages 
from Experiment 1 on a nationally representative sample in 
September 2020. Notably, in the several month period between 
Experiment 1 and Experiment 2, the public had grown increasingly 
skeptical of a potential COVID-19 vaccine [1]. Panel A of Fig. 2 plots 
the effect of each vaccine message, relative to the untreated control 
group, on the same measure of intention to vaccinate used in 
Experiment 1. (The model specifications shown in the figure were 
from our pre-registered specifications, underlying regression 
appear in Table S2.). Given that we observed the messages from 
Experiment 1 were effective at increasing vaccine uptake, we 
pre-registered directional hypotheses for Experiment 2 that tested 
whether the effects could be replicated on a nationally representa- 
tive sample. Accordingly, we report one-tailed hypothesis tests and 
90 % confidence intervals in the results presented below. Results 
largely confirmed the patterns observed in Experiment 1. 

The Baseline informational treatment was associated with a 
modest increase, 0.029 units, in intention to vaccinate (90 % CI: 
0.011, 0.046; p < .01, one-tailed test). This effect was a 6 % increase 
of the observed scale outcome in the untreated control group. 

The Community Interest and Community Interest + Embarrass 
ment messages were associated with qualitatively larger effects 
on intended vaccine uptake. These messages were associated with 
increases of 0.045 units (90 % CI: 0.021, 0.070; p < .01, one-tailed 
test) and 0.043 units (90 % CI: 0.019, 0.067; p < .01, one-tailed test), 
respectively. As with Experiment 1, we recoded those who stated 
they were “somewhat” or “very” likely to receive the vaccine as 
1 and those who did not report that they were likely to receive it 
as O (this analysis was not pre-registered: for consistency we 
report 90 % confidence intervals). This binary measure produced 
a predicted rate of intended vaccination in the control group of 
51.4 %. Intended uptake was 3.3 percentage points higher in the 
Baseline information condition (90 % CI: 0.5 pp, 6.0 pp; p < .05, 
one-tailed test), 3.5 percentage points higher in the Community 
Interest + Embarrassment condition (90 % CI: —0.1 pp, 7.0 pp; 
p =.06, one-tailed test), and 5 percentage points higher in the Com- 
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munity Interest condition (90 % CI: 1.3 pp, 0.8.7 pp; p < .05, one- 
tailed test). The latter effect was proportionally large—10 % com- 
pared to the baseline predict rate in the control group 
(0.050/0.514). 

On average, the Not Bravery, Trust in Science, and Personal 
Freedom messages were approximately as effective as the informa- 
tional content to which they were added in increasing intention to 
vaccinate, which differs from Experiment 1 where they modestly 
outperformed the Baseline informational condition. 

Turning to other regarding outcomes, Panel B of Fig. 2 plots 
effects for advice given to others and Panel C does so for negative 
judgments of non-vaccinators. The Baseline informational treat- 
ment was again associated with statistically significant increases 
in each outcome. For these outcomes, the Not Bravery, Trust in 
Science, and both Community Interest messages produced effects 
that were at least descriptively larger than the Baseline treatment. 
The effects for the Personal Freedom message were smaller than 
the Baseline informational treatment, a result that again diverged 
from Experiment 1. 

In terms of advising others to vaccinate, the most effective mes- 
sage was the Community Interest + Embarrassment message, 
which was also the most effective message in Experiment 1. This 
effect was 0.07 scale points (90 % CI: 0.043, 0.095; p < .01, one- 
tailed test), an increase of 14 % compared to the control group aver- 
age scale score of 0.51 (0.07/0.51). This effect was also statistically 
distinguishable from the effect of the Baseline informational treat- 
ment (difference = 0.045; 90 % CI: 0.020, 0.069; p < .01, one-tailed 
test). When dichotomizing the advise a friend outcome to better 
describe the magnitude of the effect, we estimated that the Com- 
munity interest + Embarrassment message was associated with a 
10 percentage point increase (90 % CI: 0.064, 0.140; p < .01, one- 
tailed test) in intention to advise others to vaccinate compared to 
the control group, a proportional increase of 27 % compared to 
the control group baseline of 38 % (0.10/0.38). This effect was 
approximately 6 points larger than the effect of the Baseline mes- 
sage (90 % CI: 0.026, 0.099; p < .01, one-tailed test). 

In terms of judging non-vaccinators, the largest effects were for 
the Not Bravery and Trust in Science messages, with each effect 
also statistically distinguishable from the Baseline message. Nota- 
bly, in this sample the Trust in Science message had large effects on 
beliefs and actions toward others but appeared ineffective in 
changing an individual’s own intended vaccination behavior. The 
Not Bravery message was also the most effective message in this 
regard in Experiment 1. 

We examined three pre-registered differences in subgroup 
treatment effects to test the patterns observed in Experiment 1. 
First, confirming Experiment 1 we found that those who did not 
endorse liberty values were more responsive to the Not Bravery 
message (compared to the baseline message) than those who 
endorsed liberty values for the three outcome measures. Second, 
we did not confirm either preregistered prediction with regard to 
differences in treatment effects by risk taking that were observed 
in Experiment 1. 

The remaining subgroup comparisons were not pre-registered. 
Beginning with gender, in comparison to the untreated control, 
women responded more to the Trust in Science and Community 
Interest + Embarrassment message than did men (all five out- 
comes), while men responded more to the Not Bravery and Com- 
munity Interest (without embarrassment) messages. Democrats 
were more responsive than Republicans across the board to the dif- 
ferent treatment messages, while Republicans appeared to react 
only to the Community Interest and Community Interest + Embar 
rassment messages (magnitudes similar to those of Democrats). 
We observed a similar pattern for differences by baseline vaccine 
confidence, measured pre-treatment with a multi-item battery of 
questions [33]. Those high in vaccine confidence responded to all 
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Fig. 2. Experiment 2. The Not Bravery, Community Interest, and Community Interest + Embarrassment messages increase both intentions to vaccinate and other-regarding 
outcomes. Panel A displays treatment effects for intentions to vaccinate, Panel B displays the advise a friend, and Panel C displays the judging a non-vaccinator outcomes. 
Treatment effects for both panels were estimated using OLS regression that included covariates. The effects displayed are a comparison against the placebo control baseline 
and are presented with 90 % confidence intervals. The dashed vertical line is the effect of the Baseline informational control for each outcome. 


messages, while those low in confidence responded reliably only to 
the Community Interest messages. 


3. Discussion 


Overall, the results point both to a set of effective messages and 
the potential efficacy of specific messages for some particular sub- 
groups. On average, a simple informational intervention is effec- 
tive, but it is even more effective to add language framing 
vaccine uptake as protecting others and as a cooperative action. 
Not only does emphasizing that vaccination is a prosocial action 
increase uptake, but it also increases people’s willingness to pres- 
sure others to do so, both by direct persuasion and negative judg- 
ment of non-vaccinators. The latter social pressure effects may be 
enhanced by highlighting how embarrassing it would be to infect 
someone else after failing to vaccinate. The Not Bravery and Trust 
in Science messages had substantial effects on other regarding out- 
comes and for some subgroups, but do not appear to be as effective 
as the Community Interest messages in promoting own vaccina- 
tion behavior. Importantly, in distinct samples fielded several 
months apart, the Community Interest, Community Interest + Em 
barrassment, and the Not Bravery messages produced substan- 
tively meaningful increases for all outcomes measures relative to 
the untreated control, and in some instances did so in comparison 
to the Baseline information condition. 

Our findings are consistent with the idea that vaccination is 
often treated as a social contract in which people are expected to 
vaccinate and those who do not are sanctioned [10]. In addition 
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to messages emphasizing the prosocial element of vaccination, 
we observed that messages that invoked reputational concerns 
were successful at altering judgment of those who would free ride 
on the contributions of others. This work could also help explain 
why social norm effects appear to overwhelm the incentive to free 
ride when vaccination rates are higher [34,35]. That is, messages 
that increased intentions to vaccinate also increased the moraliza- 
tion of non-vaccinators suggesting that they are fundamentally 
linked to one another. These messages will need to be adapted in 
specific cultural contexts with relevant partners, such as commu- 
nity leaders. 

The robust effect of the Community Interest message advances 
our current understanding of whether public health messaging 
that deploys prosocial concerns could be effective at increasing 
COVID-19 vaccine uptake. The results of both experiments pre- 
sented here support prior work that demonstrated the effective- 
ness of communication that explains herd immunity on 
promoting vaccination [15,16]. It also suggests that a detailed 
explanation of herd immunity may not be necessary to induce 
prosocial behavior. 

Beyond the theoretical contribution, the results have practical 
implications for vaccine communication strategies for increasing 
COVID-19 vaccine acceptance. We identified multiple effective 
messages that provide several evidence-based options to immu- 
nization programs as they develop their vaccine communication 
strategies. Importantly, the insights into differential effectiveness 
of various messages by subgroup (e.g. men vs women) could 
inform messaging targeted to specific groups. Understanding 
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heterogeneous treatment effects and the mechanisms that cause 
differential responses to persuasive messaging strategies requires 
additional testing and theoretical development. We view this as 
a promising avenue for future work. 

The experiments presented here are not without limitations. 
First, we measured intentions to vaccinate at a time when a vac- 
cine was not currently available and the effectiveness and side 
effects of potential vaccines were not known. This also meant that 
we could not observe actual vaccination behavior, which is ulti- 
mately the outcome of interest. While intentions predict behavior 
in many contexts [36,37] including vaccination [38-40], past 
research examining the effect of behavioral nudges on COVID-19 
vaccine uptake has produced divergent evidence when testing 
the effect of the same treatments in the field on behavior and in 
a survey experiment on a behavioral intention [41]. This observa- 
tion highlights the need for field testing messages that have shown 
to be successful on increasing uptake intentions in survey experi- 
ments to ascertain whether they also increase vaccine uptake. It 
may be that field tests reveal certain messages are particularly less 
effective than in the survey context, or that messages are uni- 
formly less effective. Second, given the rapidly evolving nature of 
the COVID-19 pandemic, attitudes about vaccines may have chan- 
ged since the experiments were fielded which could also change 
the efficacy of the messages that we tested. Third, we cannot be 
sure whether, or how long, the effects we observe here persist. 
Finally, we only tested text-based messages, but public health 
messaging is delivered through many mediums, like public service 
announcements, videos, and images. Future work can adapt the 
successful messaging strategies found here and test their efficacy 
when delivered in alternative formats. 

Efforts to vaccinate individuals against COVID-19 are currently 
underway in the United States and it remains important to con- 
vince the mass public of the safety and efficacy of COVID-19 vacci- 
nes to ensure that the threshold for herd immunity is reached. Our 
experiments provide robust evidence that appealing to protecting 
others has effects on intentions to get vaccinated and to apply 
social pressure to others to do so as well. 


4. Materials and methods 
4.1. Ethics statement 


The experiments reported here were fielded under an exemp- 
tion granted by the Yale University IRB. Informed consent was 
obtained from participants and they were informed that they could 
stop the study at any time. Data was collected anonymously and 
contained no personally identifiable information. 


4.2. Experiment 1 


Participants and Procedure. Participants were recruited by the 
vendor Luc.id to take a survey. Of those who were recruited, 
4,361 participants completed the survey. An examination of attri- 
tion during the survey reveals that attrition was balanced across 
groups which minimizes concerns that the treatment effects esti- 
mated in the main manuscript are affected by attrition. The survey 
was programmed using the survey software Qualtrics. The survey 
was fielded between July 3, 2020 and July 8, 2020. 

Experimental Design. Participants first completed basic demo- 
graphic and pre-treatment attitudinal questions and were asked 
about their experience with COVID-19. After this, participants read 
a treatment message. They were required to spend at least 20 s on 
the survey page that contained the message to given them an ade- 
quate amount of time to read it. We allocated 2/15 of the sample to 
the untreated control condition and 1/5 of the sample to the Infor- 


7164 


Vaccine 39 (2021) 7158-7165 


mation baseline condition due to the number of comparisons that 
would utilize these conditions. Each of the remaining conditions 
received 1/15 of the sample. The design and analysis were pre- 
registered at ClinicalTrials.gov (protocol ID: 2000027983). 
Outcome Measures. For COVID-19 vaccine uptake intentions, 
participants were asked “How likely are you to get a COVID-19 
vaccine within the first 3 months that it is available to you?” and 
“How likely are you to get a COVID-19 vaccine in the first year that 
it is available to you?” Respondents answered this question on a 
five-point scale with end points of “Extremely unlikely” and 
“Extremely likely.” The main text describes how these items were 
combined for analysis. Turning to the likelihood of advising some- 
one to vaccinate, respondents were asked “How likely are you to 
advise a close friend or relative to get vaccinated against COVID- 
19 once a vaccine becomes available?” Respondents also answered 
this question on a five-point scale with end points of “Extremely 
unlikely” and “Extremely likely.” Finally, for judging someone 
who chooses not to vaccinate, respondents read “we would like 


you to think about a friend or relative who chose not to receive a 
COVID-19 vaccine when it is available. What would you think 
about this person? Are they. ..”. This prompt was followed by four 
traits: trustworthy, selfish, likeable, and competent. The response 
options were “not at all”, “slightly”, “somewhat”, “mostly”, and 
“very.” 

Analysis. We used OLS regression with robust Huber-White 
standard errors and indicators for assigned treatment to estimate 
treatment effects. We use robust standard errors to address the 
heteroscedasticity observed when estimating our primary analysis 
models without them. We included covariates as described in the 
Supplementary Materials. Comparisons across treatments are from 
linear combination of coefficients tests. For the subgroup analyses, 
we restricted the sample to the stated criteria and estimate the 
model specified here on the subsample. For liberty endorsement 
and risk taking, we determined who was high and low by splitting 
the sample at the mean. 


nou 
, 


4.3. Experiment 2 


Participants and Procedure. Participants (n 5,014) were 
recruited by the vendor YouGov/Polimetrix. YouGov provides sub- 
jects using a sampling procedure that is designed to match a num- 
ber of Census demographics. To determine the sample size, we 
conducted a power analysis to detect effects that were 80 % as large 
as those observed in Experiment 1. The experiment was fielded 
between September 9, 2020 and September 22, 2020. 

Experimental Design. Participants first completed basic demo- 
graphic and pre-treatment attitudinal questions and were asked 
about their experience with COVID-19. Participants were randomly 
assigned to one of seven conditions: the untreated control, the 
Information baseline control, Community Interest, Community 
Interest + Anticipated Embarrassment, Not Bravery, Trust in 
Science, or Personal Freedom. As in Experiment 1, more partici- 
pants were assigned to the untreated control condition and the 
Baseline information control condition, 1/5 and 3/10 of the sample 
respectively. The remaining five conditions each received 1/10 of 
the sample. Participants were required to spend at least 30 s on 
the survey page that had the treatment message. The design and 
analysis were pre-registered at Open Science Framework. 

Outcome Measures. The outcome measurement was the same 
as described in Experiment 1 with the exception of intelligent 
being added to the judgment of a non-vaccinator scale. 

Analysis. We used the same modeling approach described 
above to produce the results displayed in Fig. 2. We included 
covariates as described in the Supplementary Materials. For sub- 
group analyses, we estimated OLS regression models with an indi- 
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cator variable if a person was a member of a subgroup (e.g. high 
endorsement of liberty) and zero otherwise. 
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